Statistical Learning - Final Project
Aaqil Zakarya and Jibran Haider
We start by reading in the following required libraries:
library(readr)
library(ggplot2)
library(dplyr)
library(methods)
library(stringi)
library(keras)Transfer Learning
ResNet50 and setup
We use ResNet50, which is a pre-trained neural network model with 50 trainable layers. We want to use the second last layer in the neural network (‘avg_pool’):
resnet50 <- application_resnet50(weights = 'imagenet', include_top = TRUE)
model_embed <- keras_model(inputs = resnet50$input,
outputs = get_layer(resnet50, 'avg_pool')$output)Here, we import the data for all the flower images from the following .csv and .rds files:
flowers <- read_csv("my-image-data.csv")## Parsed with column specification:
## cols(
## obs_id = col_character(),
## train_id = col_character(),
## class = col_integer(),
## class_name = col_integer(),
## path_to_image = col_character()
## )x69 <- read_rds("my-image-embed.rds")Here, we produce the embeddings:
X <- t(apply(x69, 1, cbind))
y <- flowers$class
class_names <- levels(factor(flowers$class_name))Then, we construct the training dataset using the data we have in ‘flowers’ (which we created using a 60/40 split for training and validation data respectively):
X_train <- X[flowers$train_id == "train",]
y_train <- to_categorical(flowers$class[flowers$train_id == "train"])Just to get an idea of what kind of images and their respective classes we’re looking, we print 1 sample image1 each of 18 different classes.
set.seed(1)
par(mfrow = c(3, 3))
sample_size = 18
for (j in seq_len(sample_size)) {
id <- sample(which(y == j), 1)
for (i in id) {
par(mar = rep(0, 4L))
plot(0,0,xlim=c(0,1),ylim=c(0,1),axes= FALSE,type = "n")
Z <- image_to_array(image_load(flowers$path_to_image[i], target_size = c(224,224)))
rasterImage(Z /255,0,0,1,1)
text(0.5, 0.1, label = flowers$class_name[y[i] + 1L], col = "red", cex=2)
}
}
Fitting the model
With the dataset created above, we use a neural network as the model to fit on our data (using 20 epochs). The plot shows the loss as well as the accuracy with respect to number of epochs.
model <- keras_model_sequential()
model %>%
layer_dense(units = 256, input_shape = ncol(X_train)) %>%
layer_activation(activation = "relu") %>%
layer_dropout(rate = 0.5) %>%
layer_dense(units = 256) %>%
layer_activation(activation = "relu") %>%
layer_dropout(rate = 0.5) %>%
layer_dense(units = ncol(y_train)) %>%
layer_activation(activation = "softmax")
model %>% compile(loss = 'categorical_crossentropy',
optimizer = optimizer_rmsprop(lr = 0.001 / 2),
metrics = c('accuracy'))
history <- model %>% fit(X_train, y_train, epochs = 20)
plot(history)
Results
The model fits almost perfectly for the training set (with an accuracy of around 99.5%) and quite well with the validation set (with an accuracy of around 92.7%)
y_pred <- predict_classes(model, X)
tapply(y == y_pred, flowers$train_id, mean)## train valid
## 0.9973561 0.9297066Here, we print the confusion matrix, which shows the actual value (of the class name) as compared to the predicted value for both the training and validation data sets. Each predicted value should ideally only correspond to a number of matches for the same actual value. Any matches with an actual value that is different means a case of ‘confusion’, or inaccurate prediction.
table(value = class_names[y + 1L], prediction = class_names[y_pred + 1L], flowers$train_id)## , , = train
##
## prediction
## value 0 1 10 100 101 11 12 13 14 15 16 17 18 19 2 20 21
## 0 24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 1 0 36 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 10 0 0 52 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 100 0 0 0 35 0 0 0 0 0 0 0 0 0 0 0 0 0
## 101 0 0 0 0 29 0 0 0 0 0 0 0 0 0 0 0 0
## 11 0 0 0 0 0 52 0 0 0 0 0 0 0 0 0 0 0
## 12 0 0 0 0 0 0 29 0 0 0 0 0 0 0 0 0 0
## 13 0 0 0 0 0 0 0 29 0 0 0 0 0 0 0 0 0
## 14 0 0 0 0 0 0 0 0 29 0 0 0 0 0 0 0 0
## 15 0 0 0 0 0 0 0 0 0 25 0 0 0 0 0 0 0
## 16 0 0 0 0 0 0 0 0 0 0 51 0 0 0 0 0 0
## 17 0 0 0 0 0 0 0 0 0 0 0 49 0 0 0 0 0
## 18 0 0 0 0 0 0 0 0 0 0 0 0 29 0 0 0 0
## 19 0 0 0 0 0 0 0 0 0 0 0 0 0 34 0 0 0
## 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 24 0 0
## 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 24 0
## 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 35
## 22 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 23 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 25 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 26 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 27 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 28 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## 29 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 30 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 31 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 32 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 33 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 34 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 35 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 36 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 37 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 38 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 39 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 40 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 41 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 42 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 43 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 44 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 45 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 46 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 47 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 48 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 49 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 50 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 51 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 52 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 53 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 54 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 55 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 56 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 57 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 58 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 60 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 61 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 62 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 65 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 66 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 67 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 68 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 70 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 71 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 72 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 73 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 74 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 75 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 76 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 77 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 78 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 79 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 80 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 81 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 82 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 83 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 84 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 86 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 87 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 88 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 89 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 91 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 92 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 93 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 94 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 95 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 96 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 97 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 98 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 99 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## prediction
## value 22 23 24 25 26 27 28 29 3 30 31 32 33 34 35 36 37
## 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 100 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 101 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 11 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 12 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 13 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 17 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 19 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 22 55 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 23 0 25 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 24 0 0 25 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 25 0 0 0 24 0 0 0 0 0 0 0 0 0 0 0 0 0
## 26 0 0 0 0 24 0 0 0 0 0 0 0 0 0 0 0 0
## 27 0 0 0 0 0 40 0 0 0 0 0 0 0 0 0 0 0
## 28 0 0 0 0 0 0 46 0 0 0 0 0 0 0 0 0 0
## 29 0 0 0 0 0 0 0 51 0 0 0 0 0 0 0 0 0
## 3 0 0 0 0 0 0 0 0 33 0 0 0 0 0 0 0 0
## 30 0 0 0 0 0 0 0 0 0 31 0 0 0 0 0 0 0
## 31 0 0 0 0 0 0 0 0 0 0 26 0 0 0 0 0 0
## 32 0 0 0 0 0 0 0 0 0 0 0 28 0 0 0 0 0
## 33 0 0 0 0 0 0 0 0 0 0 0 0 24 0 0 0 0
## 34 0 0 0 0 0 0 0 0 0 0 0 0 0 26 0 0 0
## 35 0 0 0 0 0 0 0 0 0 0 0 0 0 0 45 0 0
## 36 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 65 0
## 37 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 34
## 38 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 39 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 40 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 41 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 42 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 43 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 44 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 45 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 46 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 47 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 48 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 49 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 50 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 51 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 52 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 53 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 54 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 55 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 56 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 57 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 58 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 60 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 61 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 62 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 65 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 66 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 67 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 68 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 70 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 71 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 72 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 73 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 74 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 75 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 76 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 77 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 78 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 79 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 80 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 81 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 82 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 83 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 84 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 86 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 87 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 88 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 89 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 91 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 92 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 93 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 94 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 95 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 96 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 97 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 98 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 99 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## prediction
## value 38 39 4 40 41 42 43 44 45 46 47 48 49 5 50 51 52
## 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 100 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 101 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 11 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 12 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 13 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 17 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 19 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 22 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 23 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 25 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## 26 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 27 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 28 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 29 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 30 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 31 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## 32 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 33 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 34 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 35 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 36 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 37 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 38 25 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 39 0 40 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 4 0 0 39 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 40 0 0 0 76 0 0 0 0 0 0 0 0 0 0 0 0 0
## 41 0 0 0 0 35 0 0 0 0 0 0 0 0 0 0 0 0
## 42 0 0 0 0 0 76 0 0 0 0 0 0 0 0 0 0 0
## 43 0 0 0 0 0 0 56 0 0 0 0 0 0 0 0 0 0
## 44 0 0 0 0 0 0 0 24 0 0 0 0 0 0 0 0 0
## 45 0 0 0 0 0 0 0 0 118 0 0 0 0 0 0 0 0
## 46 0 0 0 0 0 0 0 0 0 40 0 0 0 0 0 0 0
## 47 0 0 0 0 0 0 0 0 0 0 43 0 0 0 0 0 0
## 48 0 0 0 0 0 0 0 0 0 0 0 29 0 0 0 0 0
## 49 0 0 0 0 0 0 0 0 0 0 0 0 54 0 0 0 0
## 5 0 0 0 0 0 0 0 0 0 0 0 0 0 27 0 0 0
## 50 0 0 0 0 0 0 0 0 0 0 0 0 0 0 154 0 0
## 51 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 51 0
## 52 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 56
## 53 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 54 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 55 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 56 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 57 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 58 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 60 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 61 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 62 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 65 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 66 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 67 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 68 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 70 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 71 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 72 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 73 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 74 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 75 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 76 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 77 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 78 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 79 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 80 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 81 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 82 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 83 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 84 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 86 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 87 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 88 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 89 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 91 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 92 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 93 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 94 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 95 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 96 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 97 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 98 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 99 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## prediction
## value 53 54 55 56 57 58 59 6 60 61 62 63 64 65 66 67 68
## 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 100 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 101 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 11 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 12 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 13 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 17 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 19 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 22 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 23 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 25 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 26 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 27 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 28 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 29 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 30 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 31 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 32 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 33 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 34 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 35 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 36 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 37 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 38 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 39 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 40 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 41 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 42 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 43 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 44 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 45 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 46 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 47 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 48 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 49 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 50 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 51 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 52 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 53 37 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 54 0 43 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 55 0 0 65 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 56 0 0 0 40 0 0 0 0 0 0 0 0 0 0 0 0 0
## 57 0 0 0 0 68 0 0 0 0 0 0 0 0 0 0 0 0
## 58 0 0 0 0 0 40 0 0 0 0 0 0 0 0 0 0 0
## 59 0 0 0 0 0 0 65 0 0 0 0 0 0 0 0 0 0
## 6 0 0 0 0 0 0 0 24 0 0 0 0 0 0 0 0 0
## 60 0 0 0 0 0 0 0 0 30 0 0 0 0 0 0 0 0
## 61 0 0 0 0 0 0 0 0 0 33 0 0 0 0 0 0 0
## 62 0 0 0 0 0 0 0 0 0 0 32 0 0 0 0 0 0
## 63 0 0 0 0 0 0 0 0 0 0 0 31 0 0 0 0 0
## 64 0 0 0 0 0 0 0 0 0 0 0 0 61 0 0 0 0
## 65 0 0 0 0 0 0 0 0 0 0 0 0 0 37 0 0 0
## 66 0 0 0 0 0 0 0 0 0 0 0 0 0 0 25 0 0
## 67 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 32 0
## 68 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 32
## 69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 70 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 71 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 72 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 73 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 74 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 75 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 76 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 77 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 78 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 79 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 80 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 81 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 82 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 83 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 84 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 86 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 87 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 88 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 89 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 91 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 92 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 93 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 94 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 95 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 96 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 97 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 98 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 99 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## prediction
## value 69 7 70 71 72 73 74 75 76 77 78 79 8 80 81 82 83
## 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 100 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 101 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 11 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 12 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 13 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 17 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 19 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 22 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 23 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 25 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 26 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 27 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 28 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 29 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 3 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
## 30 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 31 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 32 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 33 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 34 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 35 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 36 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 37 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 38 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 39 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 40 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 41 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 42 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0
## 43 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 44 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 45 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 46 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 47 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 48 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 49 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 50 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 51 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 52 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 53 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 54 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 55 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 56 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 57 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 58 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 60 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 61 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 62 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 65 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 66 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 67 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 68 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 69 37 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 7 0 51 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 70 0 0 47 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 71 0 0 0 57 0 0 0 0 0 0 0 0 0 0 0 0 0
## 72 0 0 0 0 116 0 0 0 0 0 0 0 0 0 0 0 0
## 73 0 0 0 0 0 103 0 0 0 0 0 0 0 0 0 0 0
## 74 0 0 0 0 0 0 72 0 0 0 0 0 0 0 0 0 0
## 75 0 0 0 0 0 0 0 64 0 0 0 0 0 0 0 0 0
## 76 0 0 0 0 0 0 0 0 151 0 0 0 0 0 0 0 0
## 77 0 0 0 0 0 0 0 0 0 82 0 0 0 0 0 0 0
## 78 0 0 0 0 0 0 0 0 0 0 25 0 0 0 0 0 0
## 79 0 0 0 0 0 0 0 0 0 0 0 63 0 0 0 0 0
## 8 0 0 0 0 0 0 0 0 0 0 0 0 28 0 0 0 0
## 80 0 0 0 0 0 0 0 0 0 0 0 0 0 100 0 0 0
## 81 0 0 0 0 0 0 0 0 0 0 0 0 0 0 67 0 0
## 82 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 79 0
## 83 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 52
## 84 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 86 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 87 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 88 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 89 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 91 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 92 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 93 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 94 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 95 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 96 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 97 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 98 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 99 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## prediction
## value 84 85 86 87 88 89 9 90 91 92 93 94 95 96 97 98 99
## 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 100 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 101 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 11 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 12 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 13 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 17 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 19 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 22 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 23 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 25 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 26 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 27 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 28 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 29 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 30 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 31 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 32 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 33 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 34 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 35 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 36 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 37 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 38 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 39 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 40 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 41 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 42 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 43 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 44 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 45 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 46 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 47 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 48 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 49 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 50 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## 51 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 52 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 53 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 54 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 55 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 56 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 57 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 58 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 60 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 61 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 62 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 65 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 66 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 67 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 68 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 70 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 71 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## 72 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 73 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 74 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 75 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 76 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 77 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 78 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 79 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 80 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 81 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 82 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 83 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 84 38 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 85 0 35 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 86 0 0 38 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 87 0 0 0 92 0 0 0 0 0 0 0 0 0 0 0 0 0
## 88 0 0 0 0 110 0 0 0 0 0 0 0 0 0 0 0 0
## 89 0 0 0 0 0 49 0 0 0 0 0 0 0 0 0 0 0
## 9 0 0 0 0 0 0 27 0 0 0 0 0 0 0 0 0 0
## 90 0 0 0 0 0 0 0 46 0 0 0 0 0 0 0 0 0
## 91 0 0 0 0 0 0 0 0 40 0 0 0 0 0 0 0 0
## 92 0 0 0 0 0 0 0 0 0 27 1 0 0 0 0 0 0
## 93 0 0 0 0 0 0 0 0 0 0 97 0 0 0 0 0 0
## 94 0 0 0 0 0 0 0 0 0 0 0 76 1 0 0 0 0
## 95 0 0 0 0 0 0 0 0 0 0 0 0 53 2 0 0 0
## 96 0 0 0 0 0 0 0 0 0 0 0 0 0 40 0 0 0
## 97 0 0 0 0 0 0 0 0 0 0 0 0 0 0 49 0 0
## 98 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 38 0
## 99 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 29
##
## , , = valid
##
## prediction
## value 0 1 10 100 101 11 12 13 14 15 16 17 18 19 2 20 21
## 0 12 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
## 1 0 24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 10 0 0 29 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 100 0 0 0 20 0 0 0 0 0 0 0 0 0 0 0 0 0
## 101 0 0 0 0 19 0 0 0 0 0 0 0 0 0 0 0 0
## 11 0 0 0 0 0 35 0 0 0 0 0 0 0 0 0 0 0
## 12 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 0
## 13 0 0 0 0 0 0 0 19 0 0 0 0 0 0 0 0 0
## 14 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0
## 15 0 0 0 0 0 0 0 0 0 15 0 0 0 0 0 0 0
## 16 0 0 0 0 0 0 0 0 0 0 33 0 0 0 0 0 0
## 17 0 0 0 0 0 1 0 0 0 0 0 31 0 0 0 0 0
## 18 0 0 0 0 0 0 0 0 0 0 0 0 18 0 0 0 0
## 19 0 0 0 0 0 0 0 0 0 0 0 0 0 22 0 0 0
## 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 14 0 0
## 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0
## 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 24
## 22 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 23 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 25 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 26 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 27 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 28 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## 29 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 3 0 0 4 0 0 0 0 0 0 0 0 0 0 0 2 0 0
## 30 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 31 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## 32 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 33 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 34 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 35 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 36 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 37 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 38 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 39 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 40 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 41 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 42 0 0 0 1 0 0 0 0 0 0 0 0 0 0 2 0 0
## 43 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 44 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 45 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 46 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 47 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 48 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 49 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 50 0 0 0 0 0 0 0 0 0 0 0 0 2 0 2 0 0
## 51 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 52 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 53 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 54 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 55 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 56 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 57 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 58 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 60 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 61 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 62 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 65 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 66 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0
## 67 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0
## 68 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 70 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 71 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 72 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 73 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 74 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## 75 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 76 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 77 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 78 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 79 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## 80 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## 81 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
## 82 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## 83 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 84 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 85 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 86 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 87 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 88 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 89 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0
## 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 91 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
## 92 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 93 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## 94 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## 95 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## 96 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 97 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0
## 98 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 99 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## prediction
## value 22 23 24 25 26 27 28 29 3 30 31 32 33 34 35 36 37
## 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 10 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
## 100 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 101 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 11 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 12 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 13 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 17 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 19 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 2 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0
## 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 22 36 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 23 0 17 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 24 0 0 15 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## 25 0 0 0 15 0 0 0 0 0 0 0 0 0 0 0 0 0
## 26 0 0 0 0 14 0 0 0 0 0 0 0 0 0 0 0 0
## 27 0 0 0 0 0 25 0 0 0 0 0 0 0 0 0 0 0
## 28 0 0 0 0 0 0 29 0 0 0 0 0 0 0 0 0 0
## 29 0 0 0 0 0 0 0 34 0 0 0 0 0 0 0 0 0
## 3 0 0 0 0 0 0 0 0 13 0 0 0 0 0 0 0 0
## 30 0 0 0 0 0 0 0 1 0 20 0 0 0 0 0 0 0
## 31 0 0 0 0 0 0 0 0 0 0 15 0 0 0 0 0 0
## 32 0 0 0 0 0 0 0 0 0 0 0 17 0 0 0 0 0
## 33 0 0 0 0 0 0 0 0 0 0 0 0 14 0 0 0 0
## 34 0 0 0 0 0 0 0 0 0 0 0 0 0 17 0 0 0
## 35 0 0 0 0 0 0 0 0 0 0 0 0 0 0 27 0 0
## 36 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 43 0
## 37 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 19
## 38 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 39 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 40 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 41 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 42 0 0 0 1 0 1 0 2 0 1 0 0 0 0 0 0 0
## 43 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 44 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 45 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0
## 46 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 47 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## 48 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 49 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 50 0 0 0 0 0 0 0 4 1 0 1 0 0 0 0 0 0
## 51 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
## 52 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0
## 53 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 54 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 55 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 56 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 57 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 58 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 60 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 61 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 62 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 65 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 66 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 67 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 68 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 70 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 71 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0
## 72 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 73 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 74 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 75 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 76 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 77 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 78 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 79 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 8 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0
## 80 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 81 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 82 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## 83 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 1 0
## 84 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
## 85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 86 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## 87 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 88 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 89 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## 91 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0
## 92 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## 93 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 94 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 95 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 96 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 97 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 98 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 99 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## prediction
## value 38 39 4 40 41 42 43 44 45 46 47 48 49 5 50 51 52
## 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
## 100 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## 101 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 11 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 12 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 13 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 17 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
## 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 19 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 22 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 23 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 25 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 26 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0
## 27 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 28 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 29 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 30 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 31 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
## 32 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 33 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 34 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 35 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 36 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 37 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 38 13 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 39 0 20 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0
## 4 0 0 25 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## 40 0 0 1 48 0 0 0 0 0 0 0 0 0 0 0 0 0
## 41 0 0 0 0 22 0 0 0 0 0 0 0 0 0 0 0 0
## 42 0 0 0 0 0 39 0 0 0 0 0 0 0 0 0 0 0
## 43 0 0 0 0 0 0 37 0 0 0 0 0 0 0 0 0 0
## 44 0 0 0 0 0 0 0 14 0 0 0 0 0 0 0 0 0
## 45 0 0 0 0 0 0 0 0 75 0 1 0 0 0 0 0 0
## 46 0 0 0 0 0 0 0 0 0 27 0 0 0 0 0 0 0
## 47 0 0 0 0 2 0 0 0 0 0 25 0 0 0 0 0 0
## 48 0 0 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0
## 49 0 0 0 0 0 0 0 0 0 0 0 0 36 0 0 0 0
## 5 0 0 0 0 0 0 0 0 0 0 0 0 0 18 0 0 0
## 50 0 0 0 0 0 0 0 0 0 0 0 0 0 0 86 0 1
## 51 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 32 0
## 52 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 33
## 53 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 54 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 55 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 56 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 57 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 58 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 60 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 61 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 62 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 65 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 66 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 67 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## 68 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 70 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 71 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 72 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 73 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 74 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 75 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 76 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 77 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 78 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 79 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 80 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 81 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 82 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## 83 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 84 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## 85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 86 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 87 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 88 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## 89 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 0
## 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 90 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## 91 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 92 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 93 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 94 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 95 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 96 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
## 97 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 98 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 99 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## prediction
## value 53 54 55 56 57 58 59 6 60 61 62 63 64 65 66 67 68
## 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 10 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 100 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 101 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 11 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 12 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 13 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 17 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0
## 19 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 22 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 23 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 25 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 26 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 27 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 28 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 29 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 30 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 31 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 32 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## 33 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0
## 34 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 35 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 36 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 37 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 38 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 39 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 40 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## 41 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 42 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 43 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 44 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 45 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 46 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 47 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 48 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 49 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 50 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 51 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## 52 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 53 23 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
## 54 0 28 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 55 0 0 44 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 56 0 0 0 25 0 0 0 0 2 0 0 0 0 0 0 0 0
## 57 0 0 0 0 46 0 0 0 0 0 0 0 0 0 0 0 0
## 58 0 0 0 0 0 27 0 0 0 0 0 0 0 0 0 0 0
## 59 0 0 0 0 0 0 44 0 0 0 0 0 0 0 0 0 0
## 6 0 0 0 0 0 0 0 16 0 0 0 0 0 0 0 0 0
## 60 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0
## 61 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0
## 62 0 0 0 0 0 0 0 0 0 0 22 0 0 0 0 0 0
## 63 0 0 0 0 0 0 0 0 0 0 0 21 0 0 0 0 0
## 64 0 0 0 0 0 0 0 0 0 0 0 0 41 0 0 0 0
## 65 0 0 0 0 0 0 0 0 0 0 0 0 0 24 0 0 0
## 66 0 0 0 0 0 0 0 0 0 0 0 0 0 0 15 0 0
## 67 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0
## 68 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 21
## 69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 70 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 71 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0
## 72 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 73 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 74 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 75 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 76 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 77 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 78 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 79 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 80 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 81 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 82 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 83 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 84 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 86 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## 87 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 88 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 89 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 91 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 92 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 93 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 94 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 95 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## 96 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 97 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 98 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 99 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## prediction
## value 69 7 70 71 72 73 74 75 76 77 78 79 8 80 81 82 83
## 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 10 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0
## 100 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## 101 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 11 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 12 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 13 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 15 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 17 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 18 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## 19 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 22 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 23 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 25 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 26 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 27 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## 28 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 29 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 30 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 31 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 32 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 33 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 34 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 35 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## 36 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 37 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
## 38 0 0 0 0 0 2 0 0 0 1 0 0 0 0 0 0 0
## 39 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 40 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 41 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 42 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0
## 43 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 44 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## 45 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 46 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 47 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 48 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 49 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 50 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0
## 51 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 52 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 53 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 54 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 55 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 56 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 57 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 58 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 60 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 61 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## 62 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 65 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 66 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 67 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 68 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
## 69 25 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 7 0 34 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 70 0 0 31 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 71 0 0 0 33 0 0 0 0 0 0 0 0 0 0 0 0 0
## 72 0 0 0 0 75 1 0 0 0 1 0 0 0 0 0 0 0
## 73 0 0 0 0 0 68 0 0 0 0 0 0 0 0 0 0 0
## 74 0 0 0 0 0 0 47 0 0 0 0 0 0 0 0 0 0
## 75 0 0 0 0 0 0 5 37 0 0 0 0 0 0 0 0 0
## 76 0 0 0 0 1 0 0 0 99 0 0 0 0 0 0 0 0
## 77 0 0 0 0 1 1 0 0 0 50 0 0 0 0 0 0 1
## 78 0 0 0 0 0 0 0 0 0 0 16 0 0 0 0 0 0
## 79 0 0 0 0 0 1 0 0 0 0 0 41 0 0 0 0 0
## 8 0 0 0 0 0 0 0 0 0 0 0 0 14 0 0 0 0
## 80 0 0 0 0 0 0 0 0 0 0 0 0 0 65 0 0 0
## 81 0 0 0 2 0 0 0 0 1 0 0 0 0 0 40 0 0
## 82 0 0 0 2 0 0 1 0 0 0 0 0 0 0 1 43 0
## 83 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 30
## 84 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0
## 85 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## 86 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 87 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 88 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 89 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 90 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## 91 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
## 92 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 93 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 94 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0
## 95 0 0 0 0 1 1 1 0 0 0 0 0 0 0 1 0 0
## 96 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0
## 97 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 98 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 99 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## prediction
## value 84 85 86 87 88 89 9 90 91 92 93 94 95 96 97 98 99
## 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 10 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## 100 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 101 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 11 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 12 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 13 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
## 17 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 19 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 22 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 23 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 25 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## 26 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## 27 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 28 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## 29 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 3 1 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0
## 30 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 31 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 32 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 33 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 34 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 35 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## 36 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 37 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## 38 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 39 0 0 0 0 0 0 0 0 0 0 0 1 3 1 0 0 0
## 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 40 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 41 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## 42 0 0 0 1 0 0 0 1 0 0 1 0 0 0 0 0 0
## 43 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 44 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## 45 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## 46 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 47 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 48 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 49 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 50 0 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0 0
## 51 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 52 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0
## 53 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 54 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 55 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 56 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 57 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 58 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 60 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 61 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
## 62 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 65 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 66 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 67 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## 68 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 70 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 71 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
## 72 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0
## 73 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 74 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 75 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## 76 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 77 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0
## 78 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 79 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## 80 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 81 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## 82 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0
## 83 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 84 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 85 0 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 86 0 0 23 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## 87 0 0 1 61 0 0 0 0 0 0 0 0 0 0 0 0 0
## 88 0 0 0 0 73 0 0 0 0 0 0 0 0 0 0 0 0
## 89 0 0 0 0 1 23 0 0 0 0 0 0 0 0 0 0 0
## 9 0 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 0
## 90 1 0 0 0 0 0 0 25 0 0 0 0 0 0 0 1 0
## 91 0 0 0 0 0 0 0 0 21 1 0 0 0 0 0 0 0
## 92 0 0 0 0 0 0 0 0 0 17 0 0 0 0 0 0 0
## 93 0 0 0 0 0 0 0 0 0 0 63 0 0 0 0 0 0
## 94 0 0 0 0 0 0 0 0 0 0 0 48 0 0 0 0 0
## 95 0 0 0 0 0 0 0 0 0 0 0 1 25 3 0 0 0
## 96 0 0 0 0 2 0 0 0 0 0 0 2 0 19 0 0 0
## 97 0 0 0 0 0 0 0 0 0 0 0 0 0 0 31 0 0
## 98 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 24 0
## 99 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 17The following allows us to find those examples that have the highest probability of being in a class. We use predict(mode, X) to get the probabilities and then show images that had the highest classification rate, along with their predicted class names:
set.seed(1)
par(mfrow = c(2, 3))
y_pred_mat <- predict(model, X)
id <- apply(y_pred_mat, 2, which.max)
for (i in id) try({
par(mar = rep(0, 4L))
plot(0,0,xlim=c(0,1),ylim=c(0,1),axes= FALSE,type = "n")
Z <- image_to_array(image_load(flowers$path_to_image[i], target_size = c(224,224)))
rasterImage(Z /255,0,0,1,1)
text(0.5, 0.1, label = flowers$class_name[y_pred[i] + 1L], col = "red", cex=2)
})
